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1 



(2) 
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or, 

#2G>**ry«EttU 

i: «r*H- 3 * * y ^fatti" 5 - £ 
2 ] iulHJH 1 cq^ ^ y ttf — tr>i/\?^* * 

ism i ^ ^ y ^fB«i-s^offi^sss:^-rw«-efcs 

^S^lo©^^ yfrfsf J*=*— F£> 4&<D#*y<D 

[»*S8] «!B«at«flM»»GPS'fe>'*^e>#feix5 40 
[»*B9] »*«l(MI«>ttr*-fcU, SMBS5 



tHWM%t(DZ * 7 <0&&(Djj *y\c£*)^ zm&^—tr 

amm**^ jgi©*^y we«u 
i»*3S 1 2 ] mekAi9iai{inR£ *gmk 

k* «rtB*2^y^yrtoffi«««^*2vMcBB3B#»t 

i 4 ] mm&&3fc5m&&m%mm'rz>± 

##M««i:*^IBttr*-SK2«>^^y fc, 

5] ffirlESl^^^yW:^-^^^^. 

*ycfcot, «ria4**«aw:. tirfBmi^^^y rtfc 

lie 



(3) 
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7mz&£.£tlfr*>< An — K-Cfet), «rlB«S»0*^ 

35 1 4 fc:E#<DiiMfeE«Kll. 

1 9 ] litrE* * 7 Yamm^XVt ^tio:^ 

1 4 K:E*0mGflgfiflL 

[«*S2 l ] WBtt*««ttGPS-fe^*]6»6»e>tL 
o^i *«M»i:i-SBI*35 1 4 KiEttORREftSK. 
[11*92 2] »»«P«*««>«[Sr^~fcUT, TOE 

Mjs-r -5 toes 2 (o * * v & (D&wm m t v > \z mm 

2 3 ] XSEM <C =0t5c1BMfe^m 
TOE3MSo;&* ?<d&M<d*j* 7^ J: ^ artfc*^— <3r 

£ *4W*r5BfcE«3£«. 

[«*s 2 4 1 me^AMFiafim * »»*ja*««H( 
-cisti-r ^1 4 * y zMcm~t 5 r £ t -r s» 

DW4ftSr»»L.#e>tbfcM««rE»i-5#J 

^i^/W-^LT^&UfcM^^— * 7 u A 
TOIS^-* 7 ^-A^^teg^o^TOteMiffS^ 



IWJWMlt*^ JS2«»*yfcE«i-5SR2(0>^n 

tuiemm^^ * 7 <offit <z># ;* 9 ki <t y> . «»«: 

>U*MzWi&VX±&\^fcW&(07 i -- *7 l — A£r % 

mmntm^, * 1 * » teittsK 1 ^7 

An- K#»£. 

#«P«flMBfc*R:. »2 0^^y^E«i-5*2^i3 
^ A =2 - t frEtt Lfc = ^ tr*— * ^7 A 

[*BO*»4l8B] 
[00 0 1] 

M^FI^^fi 5 fc*«>BHfc©E»*jSt»W6«K: 

A^BB-TS. *»Wf±> *?>^-bEE»o#JWS:fi l 5 = 
>-^a-^^n^7AK:E«HEfl:K:Hi-S. 
[0 0 0 2] 

■TS«ffliSIB (cyber spaces) F^^, A>5 S «V^^^]#^ 

age Based Rendering (IBR)^&§ ^S^TV^o 
[0 0 0 3] FfT% Jt^-CWfcWifcf*—^ W 

50 ^C5:H 5 T'tOo 



(4) 
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[0 0 0 4] flc»#0**^DTTH«S:«ai^«J«i" 

[0 0 0 5] IP*>> 3feW$l. 2, 3, 4t=10VvT«* 
^■ftsWMtSift/CV^. :ti6««5:IoT#^ 
"Ctf (SKI OXttSBl 10 

[0 0 0 6] SV\ ffi^^tbfdpj 

ft fc n^ft ^ *m b fr^m* z & m i 
^*60^t?9o3 6 og^eiT^^ ^^/v— Rrii^ 20 

[0 0 0 7] L^Lfcasfo, ^»©#;*7<eHB*0>SK» 

[0 0 0 8] 

fBlcfcwt* JClWft©f*— *^<— ^^MtSHIi, 

u fcsntim tut* u m^wi^- * * & 

[0009] ^oT, r«>ff!B»4*«»K:*fW»K:*u 
< . l6^Bft(^*^^^t^^tf*v^^5if^ftl*IW«:i^:^» 

fto^3aM*^^v^Tii^ftMPa^^Tt^^^fc^^^:tt. 
^tt»Mlfftfc*^Bfti<o|BJc51j^^ife^ttTU* so 



(cEt ur ^^^to fc^fctt. ±iB&ait^ 

[0 0 10] 

5 k, ^jfflfadswsostL^Mft^-^^-^^wss-r 

[0011] ±f&W!M&mj&rrz>1t*>\£. *38Ho, X 
3R»ft=»SIilftffilB*r**^5fc*©»«fci-Sfc«> 

<offittf>#;* 7lCfc 9. ^ftSrV—^r^^ir/W^aftU 
T^jSLfeMftfl)?*— ^i'— A£r. ^(Of- ^7V- 
A«:»St5»SII*i:*l^ J* 1 ©**y KJBtfcU 

WWfltiMRJ:**^ JS2fl>**y KilBftU SttT*— * 

[0012] #s»©ffaft- «*i;:«*uhu imbsii 

Oo «B©fflKt-itlcIBt»f, «WB»3£*«r±S 

gf- ^ ? u—As&mmm i > ^ y mest 5 
[0013] **w^#5gft-««fc«ixtf. Miami 

[0014] **w^»a*-HB«*-«i^rf> mrfa^^ 



(5) 



i 
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-5 Wis* 2 co * y p<9 <d$l mm * £ £51 v ^ 

[0 0 15] ^^«±iE15^feT*^Ufc®^- 

^^^nfc^^^^-r^^jft^^*^, kii 

[0 0 16] fuf5£/& 

^m#<^^^j«#^«[^u^ N swam i 

y ft^Htfgi^- 2 7 sulB^2 6o^^ey f^^e 

^Z^^na#itxf4 y {-|E^ro 0 

[0 0 17] ±ERH*»dci-Sfc*^ #»«G>. * 

fc, IBtkft ft**o^^Rj"ca»«>* ^ 7 K <t d 

* 7(D#:m<D** 7\^X V, ^w-r/WrS 

sow * £ £r*N- LTIBttrt-*SI 3 o^^en £ jmh-t z 
[0018] ^«««>»a[ft— ««fc«*ttf. itnesi 

[0 0 19] 4««©SKBft--tt*K:«ixfif, SWB*i 



w^yii, ^^^^^itetbfctf-r^-^^ey 

W<z>#»ft— ««*j:«ntf. ttE*hfe*«W\ 

[0 0 2 0] «H«>ff3Bft— ««lc*Wttf, MfE*^ 
10 7tt*H^J«S*t5. ^SW^jffaift-lB*^*!!^ 

Bffi1t**ttGPS"fe^*d*fe»6>JxS. **M©#5Bft 

^iSi-^Btrias 2 * y rto4Mtf*« * SrSv^Hii 

20 [0 0 2 1] *»W<D*fe<fc IWfii. 3e«ttft = »C7GlIHft 
ffil«ft«BK'*--5fc«)©»aii:'t-5fc«>«^ Himftftm§£ 

lco^^ey^:. buISt'— ^©«»ffiia^ov>r 

#«pja «r*rutw»*flMi a^2o^^y 
[0022] **w©»aft— ««{c«ntf v mc^K 

y^ItT-^7l/-Ai, fulE^2(^^^y p^^e 

e« « ^ v n fd H3i# it x tm-t z> n 4 <o * * y ^ 

[0 0 2 3] 

[0 0 2 4] @ 3 ^ ^ (7) t®^^^ ^>^> A co^^ ?r ^ 
so "To l^S^n^fe^^7fi7^ (2 0a, 2 0 



(6) 
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b, -20g) M^ntvNCo ^(DZj^nmzLtty 

-—ftMWy**** ^DCR-VX1000^V>fc o PC 3 0 
-fn^y^ Pentium Pro 200Hz £rF^i-o 0 PC 3 0 
x-^=^h2 1a-2 1g (y-— *tMVbox II CI 

-1100) 9@^£*i/rv>3o 

[0 0 2 5] PC 3 0 ir-O^^m — ^2 1 g ^P^tej^ 
&<7)RS232C-f >^:7 3:— ^/^f:J:oT^$ti, -Y^ 
^7x~^^-^h'2 1a, 2 1b, — 2 1 g firV 
-^ — CA*«» : VISCAIN, : VISCA 

OUT) -C»»Six-CV^5 0 S3S-T5J:5^. PC3 0fi. 
=fr^T>^7^— *3-=- y h2 1 a — 2 1 g^tT^>f 
i^^- KF^^b^if^^^^^7 2 0 a, — . 2 

i/VISCAOuT_h{-*-f Ari — KttSSr**, CW^^fAn 

— Kt»«ttpc3 o ^et>s 0 

[0 0 2 6] PC 3 0 Kite, GPSir >^ 4 0 £ 3 tt<£>^f£ 

✓<*£r:frLTPC3 (HrSWcSivcv^o tyf42d^ 
«>^ffi«»ttPC3 0rtOA/DjEftl& PFB^> iao 

tyfiOtt, TrimbletUKO W y ^ ;fr5£tf> 4 

4 0 OSrtWLfco 5 Hz 

[0 0 2 7] 4 11^ fyf^^o-;!^ 

jsv >r ± o . a -^i {cfci >t ± o . $&<Dmm 

S:««-t-S w t©-e#SDATATECHtfc»^GU-3020S:Sffl 
«-t*5T0KIN4t»«OTMC-2000SrS«bfc o GPSir V 

ttfl55tt±Sril"t-ofcfe(C. PC3 0K«tf<^ft««rJ&>l*S 

[0 0 2 8] S4Hf^ ®3H^>'*7'A-ClKft£;h,fc. 

^Bftssr&^-t-o m^<D^o^ 

^^^GPSirVim, ttfttOiSv^SS^— ^ 
Wt7 i —'?h$:tti£rt*<>o f4 Hlc^T. PC3 0rt<zv^ 

Kj tyf 4 0d^o re^T*— *j afct/te: 

[0 0 2 9] SUBKliSV^ ^-^7^7 (2 0 a — 2 0 
g) ^60*W«f4, **O^W-^2 2a l 2 
2b, 2 2g^fB^$tt5o HS^IK:, ^jAn-^ 

(20a-20g) tc^ttSfS^^^^-fo j^fcw 
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«fc5^> ;&a=:— ^ 2 0te, ^^A3-K^x4w^ 
23i, ft^-^^-^2 4 ^-^B^fc^tr-r;*- 

7-^2 2^(^1^ ^-Y An — KS?*-* V * 2 

3te. HftOy'V— A&^:^ : y^;*i--5fca&tf>^-l'A 

B* $ MfcT*— * to 7 A&g ^^^-r 6 o 
[0030] isim^ ^ a=x— yv^*\<— 

$ 2 Zfrh(0#>{ Azi — Kt, WTOCCDtyt^ 
£)<DW»X— S 4 (;J:ot^(7)7t- 

io. M:8oT7~7 , 2 2j2^fB^£*t6 0 ^6^Jd, 

^7^ 7 2 0£>^— ;7°2 2_b{Cf5^^tb^_blS2OC0v f — 
* (I«f-^t^>f^3-K) G>B»£*t5»a;<0— 

Cttlo^^f An — *ST6:h/tv^ 0 

ftfcf. fe5f- ^2 2 Jioeftioli^— *fct, -e^M 

— ^ 2 2 ±^f-^to c i: ^T*# 3 0 
[0 0 3 1 ] K^x7f-f **HDrtfc:*5rj- 

20 $r^*T 0 PC3 0te, ftSW^7ii^ttS^^T^>f 

TC60ffi N ^O^^co^JTIM. ^^^^ Aa-KtJft 
so -&m$LWf7*—?L0Z, ^^— ^POS. ^fi^— ^AZ 

[0 0 3 2] I7g|0 f^Aa-K'tyfitSVa 

— Kj d^^ibd^J: 5^, -51**1 1 x(^fi^^^^ 

— KTC ^Mfe@L0C. ^^POS. ^{iAZM^r^lSr^ 
#-C#3 0 ^^c. #bttfc^-f A=i— KTCO^b. ^ 

40 [0 0 3 3] ^3g]^y^— ^iDlftv'^T'A^g^^, ^ 

t>-t 4 o^e)^af-^T*$>^o hj^7^— ^^—^ 

Ad5B»-t-o7r^r/^5*W:. g6@, g7@W 

so [0034] ^3?!^— *m&<oum$%&m, ^im 



(7) 
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-£(DjJ ?*> "7 (20a-20g) <D^W)±^<DMM&?F 
"To 

[0 0 3 5] »8 H^fcV^THSl^SrtM^itfi 1 *!^ 
k-tZo J* 8 8, »9BWJ:}fc, ^720 c 
(#3^7^^) ^^72 0 d (#4*^7) IS*P55i 

y) W^720b (#2^7^^) li*iPtefl!l0>3&#<o 

JHBfr&SU ^7^^20e (#5#*7) W^72 10 

of (# ex*?) »*H*«o«*©JHfftfiSi- 

[0 0 3 6] ^8Bl£:fc<^-t, #7^7^^ (20g) 
«#©SIWS:»Bi-5 0 ^720g tt-toSUg*** 
^^^7 ^7 ( 2 0 a ^ 2 0 f ) CO«Kfp^ (S 8 m<D 

Sr^U3&Ay"CU^5 wi:SrBSJti-5. £fc> ^^720 

[0037] m^m&coum 
* t- a ^ .t £ M^-r— * i&m<z> & (D^kmcomti^m.m 

-T3 0 Si'oBtt, PC3 0fcJ:oTfWfP3ix5*MH« 

[0038] ^777^5 10 0T1^^7^-fi.3 
A3 — KTC^GPS-tVi^^^^^J^— 

^f^s3oom *-fAa— ktc*. # 

[0039] gn xry?s 1 0 0\^3$tfZ> 

#1#;*7 W^72 2 a) «:ffil:±X2p 

^7 7 7'S10 6m #k*^7©N^-KS:n 
^^^i"-6o **ry7S 1 0 8m JilBaf^S:Sf^0 40 

*f*H« (^-r A3-K^(Dffi3S) sr^r^-^m 

[0 0 4 0] fgl 2 0}t ^9ia<O^^^^S 2 0 OK 
fe(t5^^fA3- KTC^&i^— * TIM so 
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T'S 2 0 4m B^T*— *TIM ^fctto#1^7^7(^ 

A3 — Kd^-f « ^ *6*«2*9 (GPSir V"ir*ri> fe <om&Tl 
M) £<D^g£*n3^£2S-e#5 0 
[0 0 4 1] IP*>, gll liirgl 2®^^n— ^^r— 

o cxDmn&WLW-rZo sp^. ^f^ys 3 0 2 m 

«2TC-fe5 # 1 7^6WAn- KSrSm-TSo 

S 3 0 6"Ctt, ^fo^>^-(D' : 7—?%:W&~tZ><> XTy 
^S 3 0 8m #fc^>f A=i— K^fe^*ftiffcaB<5 
V>T. ✓n— K^cuTt^ *^HDrt<D^ A3— K • -fe^ 

[0 0 4 2] U^7El^BAXoiH» S8Hi:*5 
^T, #7#p<^ (20g) tt«*«>ai««riWi-5. - 
2 0 g^(O«K^iL^f&C0^7^7 (20a^l 

20 f) (S8HrojST-C/Tt) d»&SERIr 

fcitRtf^LTV^o *^7 2 0g«B©««i:t<: 
£K£9, * ^ 7 2 O g ^**Sr?Uj6A/-e 5 - i 
£g5ib-r£ 0 ^fc> ^^7 2 0gl:^TIforilc 

£9> &<<o%* 7 cosset ssammm-s. ip*>> 

[004 3] tt*K«©*-e3*^fc J: 5 ^F*«6» 2 

$^fAm *i 4gjtr^i-j: 5 #7 
7 (20 g ) frt>, &?m<m 
r / v (::t*v ^^M^iifl^S) 

[0 0 4 4] EP*>. 9 ^xT-k&n^L 



(8) 
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U #7^7^ 7 (2 0g) j^bOBi*^— *KlOV^C 
ti+ (r/v) 

[0 0 4 5] gll 5 0(O«yg«r, g3B©r-^iRft^ 
^fAftT9^l:H mi l0<D$O«^l@tcJ:or# 

PHflM r / v KSSi-s ^ u-Agfc 

^T-f I*7^^^77^^ (PC3 0F*3O 

HD) £/6><b. /u— jIMfe M^£> Hj^ 

[0 04 7] m3ia^->^^A«, SR6Hi:Sf7H^ 

So 

[0 0 4 8] S3'0^rAlt ^3g-r-<#^— 

tf. 8 3B©^7^lt ^l 7®tci^i-i: ^-Y 
A=z— K^— ^ TC ^ 7^— * TIM t g^&T* — ? POS <^ ^ 
lo©77^f^ (Iin^^77-1';^«) & 
W."r— * LOC £ Z TIM £ jflSM (O l o o 7 r >f 

(JIT, »2B^^77>f^S« KfEtrr^i: 5 

^*fc«B»<*n-tl^5. fc*3 % «ln^^7 7>f;^ 
l2n^77^/^(^ GPS-fc^iM O^om^)^ 40 

tSoT* ■«*IIH*rff 5 S*tS 0 
[0 0 4 9] ^Ti^^t^T-^^-^^^rA 
S3@©f- ^ilZ^V^^A^^ufcBl 7 0CO 

-/77^Att^7tfy^6 0 a — 6 0 g 

sh/cwMfc&n*. pc3 oriw-wss o ^aiR* 

*l£>o 50 
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[0 0 5 0] mi 8 0f*§H*K5 Oil^SM©*^ 
^>-^7T^/^^tf^«5 Oicfe^i-oo ^ry7S6 

oom i2n^^77Y/^tt»«5 o^eai-r 

So ^7y/S 7 0 0m ^7^^6 0/1^13 
n ^?77^^S:W»5 0 fc<E36-rS. 

[0 0 5 1 ] ^T77S8 0 0XH $ia^^77 
^;^|2D^77^f/^^ BWBt*— *TIM«r* 

o^77>f/^S3o^77^/^^ 

= — KTC^=af~^br^JS<5tt5o ^tf>P®B£Jfl2 0!§] 

[0 0 5 2] fcSS, l8@l^lLt^tfcJ:5^ 
#7^^7/&^Olt{^oV^ BMMBr/vKHBS-T 

WTfci. #7^^7i:o^t^ ^^^77^^ 

2n^^77Y/v^MltM Ur^7S8 
0 6) , f^ll, iln^77^/^|3P^ 

[0 0 5 3] ^77yS8 0 6, ^7^7 

S 9 0 6 ^«L3gSr, 121 O^Ufc^^I^^b 
T. 7^It^^»7-^^-^?:Mt 
So HP*>> 12 1 [gCD^T^^S 10 0 Otll 
^77^^11/a-K^ttitt, ^77/S10 
0 2X16^3- K^ISMt^ *TM«rfti3« 

7>7/sioo4t% #i**77bm# 6**7**1^ 

ti^2n^y^7 7>f/^5:f-^UX, ±!£TIM£ 

S 1 0 0 6T% 775/^S 1 0 0 0t*£o^ofcSln 
^77^f^V = - Kfc^y^S 1 0 0.4TEO 
^of:l2u^7 7 ^^V^ K££r»fc^ (H® 
ttfc, ««TIMi:L0Ci:3&sjK^$tt5) i"£ 0 

[0 0 5 4]-*, n^^l^oKfl ^77/S 
10 1 0T\ ^T-y^S 1 0 0 2 -C#fc^TIM 

[0 0 5 5] 

[^Cl] 

TIM - TIM - f (i) 
v 

[0 0 5 6] *rff»U ±IE*l*«>»*inM4:raU«[ 
^^r-roTIM^*-To W3i— K*r# 7^7^ 7^f2 

K«-t"SH3R«r^i-. ^^^^S 1012-C, ^7^ 
S 1 0 0 0-e^o^^ofcmi n^77^^^ 3: - 



(9) 



%Tm¥l 1-168754 



15 

Yb*y u v'7s 10 1 0T£o^ofc$2n^y^77 
[0057] 7fy/S102 om 8l u*ezs?y 

¥J»rU ftltWttoli ^77^5 1 0 2 2fcl*I 

noit#> (om&ffim * * 7 a 
j&£;ftfc 0 10 

[0 0 5 8] 7-rmOi(0&M S2 20H $8 

0^$tLfc6o<^^7^ 7 (2 0 a — 2 Of) KSSV^T 
a«P«Hftd— '«7 y 7^l*:5«A£KW^5IR3®ei-e 

5 w®**^/* (/V7^i«o^) 

5o *p«7©1Br*-C*P*:nijSi:'t-«H«*«>«JKtt5E 
^P' ttiP5*^7i:J:oTrii:«» 
S:h,S«#rttf>j&S:a*-*% £P' 7 0fjj^^^g| 20 

L (L^LO) £rt*iixTV^£-r*£. ^P* tt. P 

[0 0 5 9] 
[^2] 

[0 0 6 0] fclt^M^co^^^^C^r irtc^oo 

it. H^^^oeiF^ifi, 30 
[0 0 6 1] 
[*3] 

— K-^i)— -(¥)) 

[0 0 6 2] 1 .7 (20ai:20b, 

^fcf±2 0ci:20d, *tli20e^20f) te_bfB 

^v^^^:*t>IBttTV^S#2^p«7i:#3*7«7. St fit 
#4#^7£#5;t^7£f-oV^ff^1-3«?^ 3m^c 40 
(D^^Co^TMiiW^ g Sr*5Sfti-5 1 0 m^c 

[0 0 6 3] # I** vJb^n 6;M7fcovvCfi. ft 

D*r%*UTM»Bd«rfT5. #7#^ y (2 0 

g) j&afeoilMft^— ^Cov^tt, 7— ^Jitcjo 

i^t, mm 

ti + (r/v) 

^Mte-7— *ri s B#£(i t i^l®«7— * £: UTfE^£2xT so 
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1>3<DT\ 7*— <f»0># 7 #7* 7 (2 0 g) (CO 
l>t©I«7-^*«ffltTl«iHt5o 

[0 0 6 4] 7^W»*r-frj*"t"S*C»^ -Kffi 

J«afc 1 PBA»6niBWK:KJft't''5jMf**>S. 82 3 
0K*M- <fc 9 !6«-^rfi0o©*iRiSriRjv^2k^zM^i 

2 • 0 »©#;*7fcJ:oT«3*bfcBMfc (« : 2Z, 

t :2x) ^%^a 0 r©w«rt-ew««>*A36»fe« 

[0 0 6 5] 
1*4] 

e = tan^,[— tanej+B 0 



[0 0 6 6] k&Zo JMHcW:. iteT# > 7 0>#Eiifc* 

[0 0 6 7] H$<d** 7 - e**Ki^fe*tfciiHfcf f -: * 
*<— *SrfiLbfl>J: SJc^y 7^iB««2:*i:«>"CV^< r i: 

[0 0 6 8] l^e>R»t5S««^ ft»-fe^*36* 

•9 ri£ L#*v>^te#»*fco>-c*>3. ipt>, 8io 
H<oSJ»#ffir±8 2 4H<oSI»^J(B^2E*?5o 
S 1 1 0 0<7>$OfflIfi^7>>7 p S 1 0 O0-t*M£«Hir?<l!> 
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(57)Abstract 

PROBLEM TO BE SOLVED: To efficiently add positional 
information about a photographic position by storing a 
data frame of an image together with specific 
information in a first memory, storing its positional 
information together with acquisition time information in 
a second memory and storing the specific information 
and the acquisition time information as a pair in a third 
memory. 

SOLUTION: Time code data TC, time data TIM and 
attitude POS are stored in a first logging file and 
location data LOC and the time data are stored in a 
second logging file. Also, image data as a third logging 
file is recorded in a tape with an actual photographic 
image and the time code as a set. The first and second 
logging files are associated with the data TIM as a key, 
and also the first and third logging files are set in 
correspondence with the code TC as a key. A search 
based on a prescribed time operation is performed to 
make a future frame as object of a database. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] In order to consider as the foundation for building continuous three dimensional image 
space, it is the approach of recording the image which took a photograph with two or more 
cameras, and was obtained in two or more discrete directions of bearing. With the camera of the 
arbitration of two or more of said cameras The data frame of the image which picturized the 
object sequentially and generated it with the specific information which specifies the data frame 
Memorize in the 1st memory, acquire the positional information about the camera station of said 
data frame, and the positional information with the acquisition time information which shows the 
acquisition time of day of the positional information concerned The record approach of the image 
characterized by what it memorizes in the 2nd memory, and specific information of the data 
frame concerned and acquisition time information corresponding to the positional information 
acquired when the data frame concerned was generated are made into a pair, and is memorized 
in the 3rd memory. 

[Claim 2] Said 1st memory is the record approach of the image according to claim 1 which is 
sequential memory and is characterized by said specific information being a frame number which 
shows the record sequence of the data frame concerned in said 1st memory. 
[Claim 3] Said specific information is the record approach of the image according to claim 1 
characterized by being the information which shows the relative time of day when recording the 
data frame concerned on said 1st memory. 

[Claim 4] It is the record approach of the image according to claim 1 characterized by for said 
1st memory being the video tape memory prepared for every camera, and said 2nd memory being 
a magnetic disk. 

[Claim 5] Said specific information is the record approach of the image according to claim 1 
characterized by being the time code generated for every camera in the camera, and searching 
for the difference between the time code of this one specific camera, and the time code of other 
cameras on the basis of the time code of specific one camera of said two or more cameras. 
[Claim 6] Said camera is the record approach of the image according to claim 1 characterized by 
being carried in a car. 

[Claim 7] The record approach of the image according to claim 1 characterized by recording 
collectively the attitude information which shows the posture of a camera on said 2nd memory. 
[Claim 8] Said positional information is the record approach of the image according to claim 1 
characterized by being obtained from a GPS sensor. 

[Claim 9] The record approach of the image according to claim 1 which uses the value of 
acquisition time information as a key, and is characterized by what said the 2nd memory and said 
3rd memory are searched, and the image frame data in the 1 st [ said ] memory which the 
specific information corresponding to this acquisition time information specifies, and the 
positional information in said 2nd memory corresponding to said acquisition time information are 
associated mutually, and is recorded on the 4th memory. 

[Claim 10] The image database created by the record approach according to claim 1. 

[Claim 1 1] In order to consider as the foundation for building continuous three dimensional image 

space, it is the approach of recording the image which took a photograph with two or more 
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cameras, and was obtained in two or more discrete directions of bearing. With the camera of the 
arbitration of two or more of said cameras With the generation time information showing the 
generation time of day when the data frame was generated in the data frame of the image which 
picturized the object sequentially and generated it The record approach of the image 
characterized by memorizing in the 1 st memory, acquiring the positional information about the 
camera station of said data frame, and memorizing the positional information in the 2nd memory 
with the acquisition time information which shows the acquisition time of day of the positional 
information concerned. 

[Claim 12] The record approach of the image according to claim 1 1 characterized by what the 
value of said generation time information and acquisition time information associates mutually 
the equal image data frame in said 1st memory, and the positional information in said 2nd 
memory, and records on the 4th memory. 

[Claim 1 3] Image database system possessing an image and time-of-day database including both 
the location and time-of-day database which is the image database system which took a 
photograph with two or more cameras, and was obtained in two or more discrete directions of 
bearing, and includes both the positional information relevant to a camera station, and the 1st 
time information showing the time of day which acquired positional information, image data, and 
the 2nd time information showing the time of day which generated this image data. 
[Claim 14] In order to consider as the foundation for building continuous three dimensional image 
space, it is the recording device which records the image which took a photograph with two or 
more cameras, and was obtained in two or more discrete directions of bearing. With the camera 
of the arbitration of two or more of said cameras The 1 st memory which memorizes the data 
frame of the image which picturized the object sequentially and generated it with the specific 
information which specifies the data frame, The 2nd memory which acquires the positional 
information about the camera station of said data frame, and memorizes the positional 
information with the acquisition time information which shows the acquisition time of day of the 
positional information concerned, Image recording equipment characterized by providing the 3rd 
memory which makes a pair specific information of the data frame concerned, and acquisition 
time information corresponding to the positional information acquired when the data frame 
concerned was generated, and memorizes them. 

[Claim 15] Said 1st memory is image recording equipment according to claim 14 which is 
sequential memory and is characterized by said specific information being a frame number which 
shows the record sequence of the data frame concerned in said 1 st memory. 
[Claim 16] Said specific information is image recording equipment according to claim 14 
characterized by being the information which shows the relative time of day when recording the 
data frame concerned on said 1st memory. 

[Claim 17] It is image recording equipment according to claim 14 which said 1st memory is the 
video tape memory prepared for every camera, and is characterized by said 2nd memory being a 
magnetic disk. 

[Claim 18] Said specific information is image recording equipment according to claim 14 which is 
the time code generated for every camera in the camera, and is characterized by searching for 
the difference between the time code of this one specific camera, and the time code of other 
cameras on the basis of the time code of specific one camera of said two or more cameras. 
[Claim 19] Said camera is image recording equipment according to claim 14 characterized by 
being carried in a car. 

[Claim 20] Image recording equipment according to claim 14 characterized by recording 
collectively the attitude information which shows the posture of a camera on said 2nd memory. 
[Claim 21] Said positional information is image recording equipment according to claim 14 
characterized by being obtained from a GPS sensor. 

[Claim 22] Image recording equipment possessing the 4th memory which associates mutually the 
image frame data in the 1st [ said ] memory which uses the value of acquisition time information 
as a key, searchs said the 2nd memory and said 3rd memory, and the specific information 
corresponding to this acquisition time information specifies, and the positional information in said 
2nd memory corresponding to said acquisition time information, and memorizes them according 
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to claim 14. 

[Claim 23] In order to consider as the foundation for building continuous three dimensional image 
space, it is the storage which records the image which took a photograph with two or more 
cameras, and was obtained in two or more discrete directions of bearing. With the camera of the 
arbitration of two or more of said cameras The 1st memory memorized with the generation time 
information showing the generation time of day when the data frame was generated in the data 
frame of the image which picturized the object sequentially and generated it. Image recording 
equipment characterized by providing the 2nd memory which acquires the positional information 
about the camera station of said data frame, and memorizes the positional information with the 
acquisition time information which shows the acquisition time of day of the positional information 
concerned. 

[Claim 24] Image recording equipment according to claim 23 with which the value of said 
generation time information and acquisition time information is characterized by providing the 4th 
memory which associates mutually the equal image data frame in said 1st memory, and the 
positional information in said 2nd memory, and memorizes them. 

[Claim 25] In order to consider as the foundation for building continuous three dimensional image 
space, it is the storage of the computer program which makes a computer perform control which 
records the image which may have had the image photoed by two or more cameras in two or 
more discrete directions of bearing. With the camera of the arbitration of two or more of said 
cameras The data frame of the image which picturized the object sequentially and generated it 
with the specific information which specifies the data frame Acquire the positional information 
about the 1st program code means memorized in the 1st memory, and the camera station of said 
data frame, and the positional information with the acquisition time information which shows the 
acquisition time of day of the positional information concerned The 2nd program code means 
memorized in the 2nd memory, and the specific information of the data frame concerned, The 
storage of the computer program which memorized the 3rd program code means which makes a 
pair acquisition time information corresponding to the positional information acquired when the 
data frame concerned was generated, and is memorized in the 3rd memory. 

[Claim 26] In order to consider as the foundation for building continuous three dimensional image 
space, it is the storage of the computer program which makes a computer perform control which 
records the image which may have had the image photoed by two or more cameras in two or 
more discrete directions of bearing. With the camera of the arbitration of two or more of said 
cameras With the generation time information showing the generation time of day when the data 
frame was generated in the data frame of the image which picturized the object sequentially and 
generated it Acquire the positional information about the 1 st program code means memorized in 
the 1st memory, and the camera station of said data frame, and the positional information with 
the acquisition time information which shows the acquisition time of day of the positional 
information concerned The storage of the computer program which memorized the 2nd program 
code means memorized in the 2nd memory. 



[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the approach of adding the positional 
information about a camera station to the image sequence of the on-the-spot photo photoed 
and obtained especially with the camera efficiently about the record approach of the image for 
describing a virtual space based on an on-the-spot photo image, and equipment. Moreover, this 
invention relates to the image database system built in order to describe a virtual space based 
on an on-the-spot photo image. This invention relates to a storage at the computer program 
which controls the above-mentioned record further. 
[0002] 

[Description of the Prior Art] In recent years, the attempt which builds the urban environment 
which people gather and performs social business in the cyberspace (cyber spaces) which a 
computer network forms is performed. Usually, CG technique from the former is used for 
description and a display of this kind of virtual space; However, since a limitation is shown in CG 
expression based on a geometric model, Image Based Rendering (IBR) based on an on-the-spot 
photo image attracts attention recently. 

[0003] A photography person is experienced vicariously in reproducing in a place the image data 
obtained by the on-the-spot photo as it is. Then, based on an image database, an IBR technique 
is used, and it generates and shows on real time by making the scene of arbitration into image 
data. That is, the photoed image is treated as an image which became independent respectively, 
walk-through of an experience person's original moving trucking top for which an experience 
person will ask in a remote place if it responds for asking and rearranges can be carried out for 
it, and the virtual space of a three dimension can be sensed there. 

[0004] It is effective to use an experience person's positional information of the point where 
each image was photoed in order to have responded for asking and to have carried out retrieval 
reconstruction of the image. These display the optimal image by choosing an experience person's 
image nearest to asking from a database, and performing suitable image interpolation processing. 
Fig. 1 shows the principle of broader-based walk-through which used the on-the-spot photo 
image. 

[0005] That is, the on-the-spot photo image is prepared about narrow areas 1 , 2, 3, and 4. To 
realize broader-based walk-through (meeting a path 10 or a path 11) which continues, walks 
along these narrow areas and goes, it is necessary to interpolate the image of the space 
between narrow areas. In order to interpolate between narrow areas 2 and 3, it is necessary to 
search the on-the-spot photo image of a field 2 and a field 3 from the information about an 
experience person's between space 2 and 3 existence location. That is, in order to search a 
required image based on a user's positional information, the on-the-spot photo image needs to 
be put in a database according to the location data at the time of photography. 
[0006] Moreover, in order to interpolate with a sufficient precision and to connect smoothly the 
image and on-the-spot photo image which were interpolated, if an example is taken by there 
being walk-through possibility in [ one's / surrounding ] 360 degrees, an experience person 
needs to turn and arrange many cameras at various include angles, needs to capture an 
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environmental image to coincidence with these cameras, and needs to build an image database 
from these on-the-spot photo images. 

[0007] However, it is rather difficult to make each photography core of many cameras into one 
point, and to arrange it. He was trying to centralize the photography core of each camera on one 
point by arranging the mirror plane of each mirror so that two or more mirrors might be arranged 
to point symmetry and the light from a perimeter might go up in the former with this point, as 
shown in Fig. 2. 
[0008] 

[Problem(s) to be Solved by the Invention] However, in order to create the database of an on- 
the-spot photo image, he was trying to add positional information for each image frame of every 
in the conventional technique. However, addition in the image of this positional information 
records an image on a video tape continuously, carrying out uniform migration of the video 
camera which takes a photograph, for example, and calculates a location with the distance to the 
location on the tape which has memorized the image which is going to add positional information 
from the head of that video tape. And the positional information which photoed each image and 
image on a tape was unified, and the on-the-spot photo image database was created. 
[0009] Therefore, this activity was what needs time amount huger as it is substantially equal to 
handicraft and there are many amounts of an on-the-spot photo image. Moreover, when the on- 
the-spot photo image of a wide angle was obtained with one camera and image interpolation is 
performed using the circumference pixel of a wide angle image although an on-the-spot photo 
image must be a large area (wide angle) in order to realize broader-based walk-through, a lot of 
errors occur in the image after the interpolation, and a continuity may be substantially lost 
between a interpolation image and an on-the-spot photo image. So, when a photograph was 
taken on the spot by arranging two or more cameras (n cameras) to a wide angle, there was a 
problem said that the addition activity of the above-mentioned positional information will 
increase by n times. 
[0010] 

[Means for Solving the Problem] This invention was made in view of such a situation, and the 
purpose proposes image recording equipment in the record approach list of an image which 
closes adding the positional information about a camera station to the image sequence of the 
on-the-spot photo photoed and obtained with the camera efficiently if . Other purposes of this 
invention propose the database equipment which builds the image database with which time 
information was added so that the positional information about a camera station can be 
efficiently added to the image sequence of the on-the-spot photo photoed and obtained with the 
camera. 

[001 1] In order to consider as the foundation for building the continuous three dimensional image 
space of this invention in order to attain the above-mentioned technical problem The approach 
of recording the image which took a photograph with two or more cameras, and was obtained in 
two or more discrete directions of bearing The data frame of the image which picturized the 
object sequentially and generated it with the camera of the arbitration of two or more of said 
cameras with the specific information which specifies the data frame Memorize in the 1st 
memory, acquire the positional information about the camera station of said data frame, and the 
positional information with the acquisition time information which shows the acquisition time of 
day of the positional information concerned It is characterized by what it memorizes in the 2nd 
memory, and specific information of the data frame concerned and acquisition time information 
corresponding to the positional information acquired when the data frame concerned was 
generated are made into a pair, and is memorized in the 3rd memory. 

[0012] It is characterized by being the frame number which shows the record sequence of the 
data frame [ if it depends like 1 voice, said 1 st memory will be sequential memory, and / specific 
information / said ] concerned in said 1st memory with suitable this invention. It is the 
information which shows relative time of day with suitable this invention if it depends like 1 
voice, in case said specific information will record the data frame concerned on said 1st memory. 

[0013] It is the video tape memory with suitable this invention on which said 1st memory was 
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prepared for every camera when depending like 1 voice, and said 2nd memory is a magnetic disk. 
If it depends like 1 voice, said specific information will be the time code with suitable this 
invention generated for every camera in the camera, and will search for the difference between 
the time code of this one specific camera, and the time code of other cameras on the basis of 
the time code of specific one camera of said two or more cameras. 

[0014] If it depends on one suitable mode of this invention, said camera is carried in a car. If it 
depends like 1 voice, the attitude information with suitable this invention which shows the 
posture of a camera will also be collectively recorded on said 2nd memory. If it depends on one 
suitable mode of this invention, said positional information will be acquired from a GPS sensor. If 
it depends like 1 voice, the value of acquisition time information is used as a key, and said the 
2nd memory and said 3rd memory are searched, and the image frame data in the 1 st [ said ] 
memory which the specific information corresponding to this acquisition time information with 
suitable this invention specifies, and the positional information in said 2nd memory corresponding 
to said acquisition time information will be associated mutually, and it will record on the 4th 
memory. 

[0015] This invention also attains to the image database created by the above-mentioned record 
approach. In order to use the above-mentioned technical problem as the foundation for building 
the continuous three dimensional image space of this invention It is the approach of recording 
the image which took a photograph with two or more cameras, and was obtained in two or more 
discrete directions of bearing. With the camera of the arbitration of two or more of said cameras 
With the generation time information showing the generation time of day when the data frame 
was generated in the data frame of the image which picturized the object sequentially and 
generated it It memorizes in the 1st memory, the positional information about the camera station 
of said data frame is acquired, and it is attained also by the record approach of the image 
characterized by memorizing the positional information in the 2nd memory with the acquisition 
time information which shows the acquisition time of day of the positional information concerned. 

[0016] If it depends on one suitable mode of this invention, the value of said generation time 
information and acquisition time information will associate mutually the equal image data frame in 
said 1 st memory, and the positional information in said 2nd memory, and will record on the 4th 
memory. The above-mentioned technical problem is the image database system which took a 
photograph with two or more cameras, and was obtained in two or more discrete directions of 
bearing of this invention, and is attained by the image database system possessing an image and 
a time-of^day database including both a location and time-oi^day database including both the 
positional information relevant to a camera station, and the 1 st time information showing the 
time of day which acquired positional information, image data, and the 2nd time information 
showing the time of day which generated this image data. 

[0017] In order to consider as the foundation for building the continuous three dimensional image 
space of this invention for attaining the above-mentioned technical problem The recording 
device which records the image which took a photograph with two or more cameras, and was 
obtained in two or more discrete directions of bearing The 1 st memory which memorizes the 
data frame of the image which picturized the object sequentially and generated it with the 
camera of the arbitration of two or more of said cameras with the specific information which 
specifies the data frame, The 2nd memory which acquires the positional information about the 
camera station of said data frame, and memorizes the positional information with the acquisition 
time information which shows the acquisition time of day of the positional information concerned, 
It is characterized by providing the 3rd memory which makes a pair specific information of the 
data frame concerned, and acquisition time information corresponding to the positional 
information acquired when the data frame concerned was generated, and memorizes them. 
[0018] It is the frame number which shows the record sequence of the data frame [ if it depends 
like 1 voice, said 1 st memory will be sequential memory, and / specific information / said ] 
concerned in said 1st memory with suitable this invention. It is characterized by being the 
information which shows the suitable relative time of day of this invention if it depends like 1 
voice, in case said specific information will record the data frame concerned on said 1st memory. 
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[0019] It is the video tape memory with suitable this invention on which said 1st memory was 
prepared for every camera when depending like 1 voice, and said 2nd memory is a magnetic disk. 
If it depends like 1 voice, said specific information will be the time code with suitable this 
invention generated for every camera in the camera, and will be characterized by searching for 
the difference between the time code of this one specific camera, and the time code of other 
cameras on the basis of the time code of specific one camera of said two or more cameras. 
[0020] If it depends on one suitable mode of this invention, said camera is carried in a car. If it 
depends like 1 voice, the attitude information with suitable this invention which shows the 
posture of a camera will also be collectively recorded on said 2nd memory. If it depends on one 
suitable mode of this invention, said positional information will be acquired from a GPS sensor. If 
it depends like 1 voice, the value of acquisition time information is used as a key, said the 2nd 
memory and said 3rd memory are searched, and the 4th memory which associates mutually the 
image frame data in the 1st [ said ] memory which the specific information corresponding to this 
acquisition time information with suitable this invention specifies, and the positional information 
in said 2nd memory corresponding to said acquisition time information, and memorizes them is 
provided. 

[0021] In order that other purposes of this invention may consider as the foundation for building 
continuous three dimensional image space It is the recording device which records the image 
which took a photograph with two or more cameras, and was obtained in two or more discrete 
directions of bearing. With the camera of the arbitration of two or more of said cameras The 1st 
memory memorized with the generation time information showing the generation time of day 
when the data frame was generated in the data frame of the image which picturized the object 
\ sequentially and generated it, The positional information about the camera station of said data 
frame is acquired, and it is attained by the image recording equipment characterized by providing 
the 2nd memory which memorizes the positional information with the acquisition time information 
which shows the acquisition time of day of the positional information concerned. 
[0022] If it depends like 1 voice, the 4th memory with suitable this invention which associates 
mutually the image data frame in said 1st memory with the equal value of said generation time 
information and acquisition time information and the positional information in said 2nd memory, 
and memorizes them is provided. 
[0023] 

[Embodiment of the Invention] Hereafter, the collecting system of the image which applied this 
invention, and the system which builds a database from the image which is an image database for 
making broader-based walk-through possible further, and were collected by this collecting 
system are explained. 

< Configuration of an image acquisition system> This image acquisition system photos a 
circumference environment with two or more cameras carried in the car. Moreover, the attitude 
sensor which detects the posture (namely, posture of a camera) of a car body, and the bearing 
sensor which detects bearing with earth magnetism are carried in the GPS sensor list which 
detects a car location at the car. 

[0024] The configuration of this image acquisition system is shown in Fig. 3. Seven-set (a 
[ 20 ],b [ 20 ], — 20g) loading of the camera for photoing an environment is carried out. This 
camera used video camera DCR-VX1000 by Sony Corp. PC30 controls this whole image 
acquisition system, and contains microprocessor Pentium Pro 200Hz. They are video / computer 
interface unit 21a between PC30 and each camera 20. — 21 g (Vbox II CI[ by Sony Corp. ]- 
1 100) connects. 

[0025] It connects with a well-known RS232C interface bus between PC30 and interface 21 g, 
and connects by the daisy chain mode (input signal: VISCAIN, output signal:VISCAOUT) interface 
units 21a and 21 b and — 21 g. it mentions later — as — PC30 — each interface unit 21 — a 
time code inquiry signal is sent to each cameras 20a, — , 20g through a — 21 g. Each cameras 20a, 
— , 20g carry time code information on signal-line VISCAOUT, and this time code information 
gets across to PC30. 

[0026] The GPS sensor 40 and the attitude sensor 41 of three shafts are connected to PC30 
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through each RS232C bus interface, and the earth magnetism bearing sensor 42 is connected to 
PC30 through the analog bus. A/D conversion of the bearing signal from a sensor 42 is carried 
out by the A/D converter in PC30 (un-illustrating). 4400 of the kinematic method made from 
Trimble was used for the GPS sensor 40 used for this system. The sensor of a pneumatic 
method enables location measurement in the precision of **3cm with the sampling rate of 5Hz. 
[0027] GU-3020 made from DATATECH which can secure the precision of **0.9 degrees in the 
precision of **0.5 degrees and a yaw angle in a pitch angle and a roll angle were adopted as the 
attitude sensor 41. Moreover, TMC-2000 made from TOKIN which secure *#two precision were 
adopted as the earth magnetism bearing sensor 42. In addition, the activity which calculates 
positional information based on the signal from the GPS sensor 40 has a possibility of covering 
too much load over PC30, in order to require rapidity. Then, PC only for GPS data operations 
may be added as deformation of the data collecting system of Fig. 3. 

[0028] The record location of the various data collected by the system of Fig. 3 is shown in Fig. 
4. As everyone knows, the GPS sensor of a kinematic method outputs the high time-of-day data 
and location data of precision. In Fig. 4, "positional information data", the "attitude information 
data" from an attitude sensor 41 , and the "bearing information" from the bearing sensor 42 are 
remembered to be the "time codes" from a camera 20 from a sensor 40 to a "time-of-day data" 
list by the hardware disk HD in PC30. 

[0029] In Fig. 4, the on-the-spot photo image from each camera (20a — 20g) is recorded on 
each video tapes 22a, 22b, — , 22g. The method of the record in a camcorder (20a — 20g) is 
shown in Fig. 5. As everyone knows, a camcorder 20 contains the time code generator 23, the 
signal mixer 24, and the video tape 22 for data logging. The time code generator 23 outputs the 
time code for carrying out the index of the frame of an image. That is, a time code pinpoints the 
frame location of the image data recorded on the tape. 

[0030] In Fig. 5, the time code from the time code generator 23 and the image data from a non- 
illustrated CCD sensor are recorded on a tape 22 according to a predetermined format by the 
mixer 24. An example of the format that the two above-mentioned data (image data and time 
code) recorded on the tape 22 of a camera 20 are recorded on Fig. 6 is shown. That is, one time 
code is assigned to each image frame on the tape. If it puts in another way, the image data of 
the purpose on a certain tape 22 can be searched on the tape 22, if the time code corresponding 
to the image data is specified. 

[0031] An example of a record format of one record of said various information in the hardware 
disk HD is shown in Fig. 7. If a time code is received from the camera of arbitration at the time 
of arbitration, PC30 will carry out "bearing information" to the "time-of-day data" (it received 
from the sensor 40) at that time, the "positional information" (it received from the sensor 40) of 
a car, and the "attitude information" from an attitude sensor 41 from the bearing sensor 42 at a 
group, and will write it in in Disk HD as a time code sensor information record file (format of Fig. 
7). That is, one time code sensor information record is constituted by the car location data LOC, 
the posture data POS, the bearing data AZM, etc. at the time of receiving number [ of the 
camera which generated the value of a time code TC, the time of day TIM at that time, and its 
time code ] #, and its time code. 

[0032] The record which the value of a certain time of day tx is given, and has TIM near the 
value of the time of day tx from the value of *# and its time of day tx is found, and a time code 
TC, the car location LOC, Posture POS, and Bearing AZM can be known from the record so that 
clearly from the "time code sensor information record" of Fig. 7. Moreover, a tape 22 top can be 
searched and the target image data can be obtained from the value of the obtained time code 
TC. 

[0033] The purpose of the data collecting system of Fig. 3 is to generate the image data which 
becomes a radical for generating a convenient image database although a panorama image is 
generated. Here, the signal with which the location of sensor data or image data can be 
expressed is time-of-day data from a sensor 40 in a time code and a sensor signal with a 
camera. Finally as for an image database, image data, and the positional information and posture 
data of a car should just be connected. Therefore, file format which data collecting system 
memorizes can be considered as the format adapted to the amount of data also except Fig. 6 
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and the 7th Fig. 

[0034] Collection of the below-mentioned data image changes the acquisition method of Fig. 6 
and Fig. 7. 

<Arrangement of a camera) Arrangement of a up to [ the car of seven cameras (20a-20g) of this 
data collecting system ] is shown in Fig. 8. 

[0035] Let the drawing upper part be the travelling direction of a car in Fig. 8. As shown in Fig. 8 
and Fig. 9, it takes charge of photography of the environment of a car travelling direction camera 
20c (#3 camera) and camera 20d (#4 camera), and camera 20a (#1 camera) and camera 20b (#2 
camera) take charge of photography of the environment on the left-hand side of a car, and take 
charge of photography of the environment on the right-hand side of a car camera 20e (#5 
camera) and camera 20f (#6 camera). 

[0036] In Fig. 8, #7 camera (20g) photos a back environment. The photography core has 
estranged only distance r camera 20g from the photography core (shown in respect of [ T ] Fig. 
8) of other cameras (20a thru/or 20f). By putting camera 20g on the posterior part of a car, it 
prevents that camera 20g copies a car body. Moreover, the degree of freedom of arrangement of 
many cameras is secured by lowering camera 20g back. That is, it prevents that many cameras 
focus on a center position T too much. 

[0037] <Collection of an on-the-spot photo image> Next, the flow of processing for the image 
data collection by this data collecting system is explained. Fig. 10 explains the overall procedure 
of the on-the-spot photo image data collection processing controlled by PC30. 
[0038] The time codes of all cameras are matched at step S100. At step S200, matching with a 
time code TC and the time-of-day data from a GPS sensor is performed. At step S300, 
matching with a time code TC and the information from each sensor is performed. Processing of 
step S300 is repeated until photography is completed. 

[0039] Fig. 11 shows the detailed procedure of "matching of time code" processing in step S100. 
That is, the value of the number **** counter k of a camera is set to 2 at step S1 02. The 
reason set to 2 is that it is only making #1 camera (camera 22a) into criteria for convenience in 
this system. So, at step S104, logging of the time code from #1 camera is carried out. At step 
S106, logging of the time code of #k camera is carried out At step S108, it judges whether only 
the count of predetermined repeated the above-mentioned actuation. By this actuation, two or 
more data in which the correspondence relation (difference of the value of a time code) in 
combination with the time code of the camera k of the time code of #1 camera and the 
arbitration of #2 camera thru/or the #7 cameras is shown were obtained. If two or more of these 
data are equalized, matching with the time codes of #1 camera and #k camera, i.e., a difference 
of a value, will be obtained. This matching is performed about all the cameras of #2 to #7 camera. 

[0040] Fig. 1 2 is the detail of the procedure for obtaining matching with the time code TC and 
the time-of-day data TIM in step S200 of Fig. 9. That is, the time-ol^day data TIM from a GPS 
sensor are obtained at step S202. At step S204, it is the time-of-day data TIM. Logging of the 
value of the time code TC of #1 camera which can be set is carried out The above-mentioned 
actuation can be repeated several times and a difference with time of day (time of day TIM from 
a GPS sensor) can be absolutely known from the average with the value which the time code in 
#1 camera shows. 

[0041] That is, with the relation obtained by the flow chart of Fig. 1 1 and Fig. 1 2, when the time 
code of a certain camera k is TCk, the time code of #1 camera is how many, and it can grasp 
beforehand how much it is shifted in the time interval. Fig. 1 3 explains the detail of step S300. 
That is, at step S302, the time code from #1 camera which is criteria is received. At step S304, 
the posture data (a pitch angle, a roll angle, yaw angle) from an attitude sensor are memorized. 
The data of a bearing sensor are acquired at step S306. At step S308, it records on the time 
code sensor information file in the hardware disk HD by one record based on a time code, a 
sensor, etc. which were obtained. 

[0042] <Reservation of the degree of freedom of camera arrangement> In Fig. 8, #7 camera (20g) 
photos a back environment The photography core has estranged only distance r camera 20g 
from the photography core (shown in respect of [ T ] Fig. 8) of other cameras (20a thru/or 20f). 
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By putting camera 20g on the posterior part of a car, it prevents that camera 20g copies a car 
body. Moreover, the degree of freedom of arrangement of many cameras is secured by lowering 
camera 20g back. That is, "rt prevents that many cameras focus on a center position T too much. 

[0043] As the term of the conventional technique described, in order to generate one 
interpolation image, if these camera stations are estranged too much, the smoothness of them 
will be lost from the on-the-spot photo image photoed at 2 points of discontinuity between a 
interpolation image and an on-the-spot photo image. Then, as this system shows to Fig. 14, it is 
time amount width-bf-face r/v from #7 camera (20g). (it is here and v is the speed of advance of 
a car) 

#1 thru/or #6 camera decides to use with the image data to which only r/v time amount took a 
photograph of the on-the-spot photo image data photoed when the image data of the **** 
future, i.e., #7 camera, progressed after r/v time amount in a location T on the spot in the past 
location (namely, location shown in Fig. 14). 

[0044] In order to realize a broader^based walk-through system, namely, the image database 
which serves as a radical for generating a interpolation image from an on-the-spot photo image 
# Although the on-the-spot photo image of one camera thru/or #7 camera must be connected 
mutually and it must consider as a database, as shown in Fig. 1 5 # The image of one camera 
thru/or #6 camera unifies the images (namely, the images which have the same "time-of-day 
data") photoed at the actually same time of day (it considers as time of day t1) as one record of 
a database, and is time-of-day t1+ (r/v) about the image data from #7 camera (20g). 
*5Mc******3M=* j s unified on this record. 

[0045] In processing Fig. 1 5 with the data collecting system of Fig. 3, it corrects the relation 
corresponding to the above by subtracting only the frame number equivalent to time amount 
width-ofHace r/v further to the correspondence relation of the time code of #1 camera of a 
place and the time code from #7 camera which were obtained by the control procedure of Fig. 
1 1 . That is, only the frame number to which the time code of #7 camera is always equivalent at 
above-mentioned time amount width-of-face r/v will show the image data of the future so much 
to the time code of #1 camera thru/or #6 camera. 

[0046] <Database generative system> Fig. 1 6 shows the database generation structure of a 
system which generates the database which was suitable for performing image interpolation for 
broader^based walk-through image presentation from the video tape file obtained by the data 
collecting system of Fig. 3, and the magnetic disk file (HD in PC30). 

[0047] The system of Fig. 3 generates two files shown in Fig. 6 and Fig. 7. Two files are possible 
for making the location of the camera when photoing the image data for the image data of the 
frame of arbitration, the posture of a camera, and bearing of a camera link with a time code. The 
need for a database generative system is in the place which moves and replaces the image data 
on a tape with a high-speed file. 

[0048] Moreover, since 3rd [ ** ] Fig. system has much amount of data which should be 
processed, it may be difficult to generate the time code sensor information file of Fig. 6. If it puts 
in another way, he is trying for the time code data TC, the time-o^day data TIM, and the 
posture data POS to memorize the system of Fig. 3 to one file (for it to be called the 1 st logging 
file) at one file (for it to be hereafter called the 2nd logging file) with another location data LOG 
and time-of-day data TIM, as shown in Fig. 1 7. Moreover, as the 3rd logging file, an on-the-spot 
photo image and a time code become a group, and image data is recorded into the tape. In 
addition, since the record is generated whenever the output of the GPS sensor 40 generates the 
1 st logging file and the 2nd logging file, there is much amount of data, therefore the precision at 
the time of performing image interpolation is secured. 

[0049] The database generative system explained below changes into a database two logging 
files of Fig. 1 7 which the data collecting system of Fig. 3 generated. Generation of a database is 
performed by the image processing system computer 50 in Fig. 16. The above-mentioned video 
tape file is set to videocassette recorders 60a-60g, and is read. Moreover, PC30 is connected to 
a computer 50. 

[0050] On the whole, Fig. 18 shows the flow of processing by the computer 50. Step S500 
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transmits the 1st logging file to a computer 50 from PC30. At step S600, the 2nd logging file is 
transmitted to a computer 50. At step S700, the 3rd logging file is transmitted to a computer 50 
from a videocassette recorder 60. 

[0051] At step S800, the time-of-day data TIM are used as a key, and the 1st logging file and 
the 2nd logging file are matched. The detail is shown in Fig. 19. The result of this processing is 
memorized by the hardware disk in a calculating machine 50 (un-illustrating). At step S900, a 
time code TC is used as a key and the 1 st logging file and the 3rd logging file are matched. The 
detail is shown in Fig. 20. The result of this processing is also memorized by the hardware disk in 
a calculating machine 50 (un-illustrating). 

[0052] in addition, as explained in relation to Fig. 8, about the processing which sets the frame of 
the future equivalent to time amount width-of-face r/v as the object of a database about the 
image from #7 camera # Matching processing of the 1 st logging file about seven cameras, and 
the 2nd logging file (step S806), What is necessary is just to perform the search based on the 
time amount actuation corresponding to above-mentioned r/v in a list about matching 
processing (step S906) of the 1 st logging file and the 3rd logging file. 

[0053] It specifically deforms into the control procedure which showed processing of step S806 
and step S906 in Fig. 21, and the image database for panorama image composition is created. 
That is, one record of the 1st logging file is read at step S1000 of Fig. 21, and the time-of-day 
data TIM in the record concerned are got to know at step S1002. Next, about the image data of 
#1 camera thru/or #6 camera, it is step S1 004, and the inside of the 2nd logging file of #1 
camera thru/or each #6 camera is searched, and the record which has TIM which has the same 
value as Above TIM is looked for. At step S1 006, the record of the 1 st logging file found at step 
S1000 and the record of the 2nd logging file found at step S1004 are unified (Information TIM 
and LOC is unified substantially). 

[0054] time of day TIM which is step S1010 and was obtained at step S1002 about #7 camera on 
the other hand [from — 0055] 
[Equation 1] 
TIM = TIM - f (-) 



[0056] It calculates and the record which has TIM which has the same value as the time of day 
TIM of the number 1 above-mentioned formula is looked for in the 2nd logging file of #7 camera. 
In addition, f in the above-mentioned formula shows the function changed into the time amount 
width of face for which time amount width-of-face r/v is needed in this data collection system. 
At step SI 01 2, the record of the 1 st logging file found at step S1000 and the record of the 2nd 
logging file found at step S1 01 0 are unified. 

[0057] At step S1020, if it judged whether the above-mentioned processing was performed and 
ended about all the records of the 1st logging file, it will progress to step S1022 and the record 
which unified the 1 st logging file and the 2nd logging file will be written in a database file. In this 
way, the database system with the easy image interpolation for performing broader-based walk- 
through in a calculating machine 50 was generated. 

[0058] <Generation of a panorama image> Fig. 22 is a principle Fig. explaining the include angle 
which makes the visual field range overlap in six cameras (20a-20f) shown in Fig. 8. Generally, in 
interpolation of the image obtained from two cameras (whenever [ angle-of-visibility ] is the 
same) with which only the distance of 2d has been estranged and arranged, the case where an 
image is compounded so that the object P in distance L0 may be connected smoothly 
(generation of a panorama image) is considered now. The field of the triangle which makes Point 
P top-most vertices ahead [ of a camera ] serves as a dead angle. Moreover, point P' shows the 
point in the field picturized by the duplex with both cameras. Supposing it is separated from 
point P' of the distance L ahead of a camera (L>=L0), point P' is [0059] from P. 
[Equation 2] 
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[0060] It will be reflected on the central site of a ****** image. Therefore, when the image of 
two sheets is connected, for Object P, the difference between the locations in which it is 
displayed on a duplex and the image of a duplex is reflected is [0061]. 
[Equation 3] 

[0062] It becomes. 1 set of cameras (20a, 20b, 20c, 20d, or 20e and 20f) set up an overlap 
include angle in consideration of the above-mentioned difference. If it calculates about #2 
camera from which are most separated of the photography core, #3 camera or #4 camera, and 
#5 camera and the joint of an image will be optimized about the object of 3m beyond, the object 
of 10m beyond means being reflected in a duplex by the angular difference whose object of the 
method of infinite distance is 9.3 degrees 6.5 degrees (it converts into distance and is 1.1m). 
[0063] # About one camera thru/or #6 camera, since the photography core is close, perform 
image interpolation in consideration of the duplex projection in consideration of the above- 
mentioned formula. Moreover, about the image data from #7 camera (20g), it sets on a database, 
and is time-of-day t1+ (r/v). 

Since ************ j s recorded as image data of time of day t1, image interpolation is carried 
out using the image data about #7 camera (20g) in a database. 

[0064] In order to compound the panorama image of one sheet, it is necessary to change virtual 
plane of projection into a cylinder side from a flat surface. As shown in Fig. 23, the image (length: 
2Z, horizontal:2X) photoed with the camera of horizontal-angles-of-view 2 and thetaomega 
which turned to the direction of bearing theta 0 absolutely is considered. The projection location 
(theta, z') to the flat surface at the time of projecting the image which is in the location of 
Length z and Width x from the center of an image within this image to up to cylinder plane of 
projection is [0065]. 
[Equation 4] 

8 = tan_,[ — tan9 ) + 0o 



[0066] It becomes. The image which performed spherical-aberration amendment of a camera 
first, performed the above-mentioned conversion in fact, and was projected on the cylinder side 
was obtained, and the panorama image of one sheet has been obtained by arranging this in a 
horizontal single tier. Moreover, blending processing is performed into the lap part of images, and 
a continuous change is given. 

[0067] By packing into a panorama image the image database made in large quantities with seven 
cameras as mentioned above, the database of a panorama image is newly done. This serves as a 
source image used by this walk-through system. 

Modification of data collection> — The data collection shown in Fig. 10 and Fig. 11 the 1st 
modification was effective when the output from an attitude sensor was irregular. 
[0068] The modification to be explained from now on is made a period with the fixed output from 
an attitude sensor, and when [ which is the data ] it spills picking and there is nothing **, it is 
effective. That is, the control procedure of Fig. 10 changes to the control procedure of Fig. 24. 
Control of step S1 1 00 is equivalent to it of step S1 00. Moreover, control of step S1 200 is 
equivalent to control of step S200. 

[0069] At step S1300, matching with the output POS of a time code TC and an attitude sensor 
is performed. Specifically logging of the output data of an attitude sensor is carried out, logging 
of the time code of#step S1 camera is carried out by 1304 in step S1 302 of Fig. 25, these are 
made into a pair, and it writes in a disk. At step S1400 of Fig. 24, matching with output-data 
POS of an attitude sensor and the output AZM of a bearing sensor is performed. Logging of the 
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output of an attitude sensor is carried out at step S1 402 of Fig. 26, and, specifically, logging of 

the output of a bearing sensor is carried out at step S1404. In this way, an attitude sensor 

output and a bearing sensor output serve as a group, and logging is carried out 

[0070] By the technique of Fig. 24, in the loop formation between step S1 500 and step S1400, 

the need of taking a time code into consideration is lost, and data collection becomes a high 

speed. 

Modification of camera arrangement> — In arrangement of the camera shown in Fig. 8 the 2nd 
modification, only one was installed camera 20g which photos a back background. In order to 
cover a back field of view with the camera of a piece, although it is necessary to set up a 
camera 20g angle of visibility widely, if a visual field is set up widely, surrounding resolution may 
deteriorate and the relation in the case of image interpolation may become discontinuity. 
Moreover, although camera arrangement of Fig. 8 does not have a problem at the time of 
rectilinear propagation, a problem occurs at the time of a right and left chip box. 
[0071] The modification shown in Fig. 27 increases one #8 camera back. By having set the back 
camera to two, the visual field of one camera (#7 or #8) becomes it good that it is narrow. 
Moreover, in order to narrow the field used as a dead angle, the optical axis of #7 camera and #8 
camera is made to cross, as shown in Fig. 27. When an image acquisition system is carried in the 
car of camera arrangement of Fig. 27 and this car goes straight on, after r/v time amount, #7 
camera and #8 camera move in the location of T of Fig. 28, and 8', respectively. That is, when a 
car continues advance, the image data of both #7 camera 7 in front of the frame number 
equivalent to r/v time amount and #8 camera is used. 

[0072] # One of the advantages which makes seven cameras and #8 camera cross appears at 
the time of right-turn or left turn. That is, in left turn, since a car body is rotated 
counterclockwise, #8 back camera arrives at an arrangement location (8* location) as shown in 
Fig. 29. That is, suppose that the image data of #1 camera when being in the image data of #8 
camera in 8' location and the location before left turn thru/or #6 camera is combined. If it does 
in this way, image data when #1 camera thru/or #6 camera turn to the direction (the rectilinear- 
propagation direction) of [ before a car body turning left ] will be unified with image data when #8 
camera is suitable in the rectilinear^propagation direction, and the problem of arrangement of 
Fig. 8 will be solved. 

[0073] In addition, detection of left turn and right-turn can be known from the output of 
************ 41 That is, a computer 50 chooses the image data from #7 camera (or #8 
camera), when the file the output of an attitude sensor 41 is remembered to be is read and the 
turn direction at that time shows right-turn (or left turn). If a car body's turning clockwise 
(counterclockwise rotation), speaking generally, the output of the camera formed by shifting 
counterclockwise (clockwise rotation) rather than the center line will be adopted. 
[0074] In right-turn, Fig. 30 shows that the image data of #7 camera is used. In addition, the 
control procedure of the 2nd modification becomes naturally clear from the control procedure of 
the 3rd below-mentioned modification. 

Modification of camera arrangement> — The 3rd modification (Figs. 31 - 34) of the 3rd 
modification adds further #8 camera which turns to the direct back of a car to the 2nd 
modification. 

[0075] According to this arrangement, at the time of rectilinear propagation, as shown in Fig. 32, 
as shown in Fig. 33 at the time of left turn, as shown in Fig. 34, at the time of right-turn, the 
image data of #7 camera arranged on the left-hand side of back is used using the image data of 
#9 camera arranged on the right-hand side of back using the image data of #8 central camera. 
The 3rd modification is effective in relation of an image becoming smooth further rather than the 
2nd modification. 

[0076] The procedure of database generation of the 3rd modification is shown in Fig. 35. This 
control procedure is substantially [ as the 2nd modification ] the same at the point which image 
data of the cameras which photo a back field of view based on the travelling direction (it can 
judge from the posture data POS or the bearing data from a sensor 42) of a car to use. A 
different place from it of Fig. 21 is the posture data POS (or) at step S2010. Based on bearing 
data, right-turn, rectilinear propagation, or left turn is judged from a sensor 42. In right-turn # 
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Adopt the image data of seven cameras (Fig. 33), in rectilinear propagation, adopt the image data 
of #8 camera (Fig. 31), and, in left turn, adopt the image data of #9 camera (Fig. 32). 
[0077] <Synchronization of photography and data base construction> — With the 4th 
modification above-mentioned implementation gestalt, the image database was built after 
photography based on the image data beforehand photoed on the tape. However, it is also 
possible to build a database, taking a photograph. In this case, large capacity and high-speed file 
equipment are needed. 

[0078] in addition, in building the real time of a database to the above-mentioned operation 
gestalt shown in Figs. 14 or 27 or a **** 31 Fig. The buffer with which only the above-mentioned 
time amount f (r/v) delays the image data which the camera (8 7 the example of 14 Fig. camera 
# the example of 27 Fig. camera # 7 and # the example of 31 Fig. # 7 thru/or # 9 cameras) 
arranged in the aft position photoed is needed. 

[0079] <Other modifications> With the above-mentioned operation gestalt, as shown in Fig. 1 7, 
the combination of the data memorized to the 1 st logging file and the 2nd logging file recorded 
TC, TIM, and POS on the former, and recorded LOC and TIM on it at the latter. However, the 
combination of record is not limited to the combination of Fig. 17. 

[0080] That is, if an example is taken by image data being large capacity, as for image data, it is 
desirable to record on bulk memories, such as a magnetic tape, independently. However, about 
TC, TIM, POS, and LOC, you may record on one high-speed file. 
[0081] 

[Effect of the Invention] As explained above, according to this invention, it becomes possible to 
add the positional information about a camera station to the image sequence of the on-the-spot 
photo photoed and obtained with the camera efficiently. Moreover, the database equipment 
which builds the image database with which time information was added is made so that the 
positional information about a camera station can be efficiently added to the image sequence of 
the on-the-spot photo photoed and obtained with the camera. ~\ 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
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1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] Drawing explaining the outline of a broader-based walk-through method in which this 
invention may be applied. 

[Drawing 2] The side elevation showing the configuration of the outline of camera loading 
equipment prepared in conventional camera arrangement equipment. 
[Drawing 3] The block diagram showing the configuration of the data collecting system 
concerning the example which applied this invention. 

[Drawing 4] Drawing showing where various data are finally recorded in the system of Fig. 3. 
[Drawing 5] The block diagram showing generation of the time code in the video camera 20 of 
Fig. 3. 

[Drawing 6] Drawing explaining a format of the data recorded on the video tape 22 of a video 
camera 20. 

[Drawing 7] Drawing explaining a format of the data recorded in the hardware disk HD of PC30. 
[Drawing 8] Drawing explaining the description of camera arrangement of this operation gestalt. 
[Drawing 9] Drawing which explains photography bearing of #1 camera thru/or #6 camera in 
camera arrangement of Fig. 8. 

[Drawing 10] The flow chart which shows the whole processing of the data collecting system of 
Fig. 3. 

[Drawing 1 1] The flow chart which explains a part of flow chart of Fig. 10 to a detail. 
[Drawing 12] The flow chart which explains a part of flow chart of Fig. 10 to a detail. 
[Drawing 13] The flow chart which explains a part of flow chart of Fig. 1 0 to a detail. 
[Drawing 14] Drawing explaining the advantage of camera arrangement of Fig. 8. 
[Drawing 15] Drawing explaining the time of day of the image data unified on one record of an 
image database. 

[Drawing 16] The block diagram showing the database generation structure of a system of an 
operation gestalt. 

[Drawing 17] Fig. 6, drawing showing the configuration of files other than the format of Fig. 7. 
[Drawing 18] The flow chart explaining radical Motohara ** of control for database generation of 
an operation gestalt. 

[Drawing 19] The flow chart which explains a part of flow chart of Fig. 1 8 to a detail. 

[Drawing 20] The flow chart which explains a part of flow chart of Fig. 18 to a detail. 

[Drawing 21] The flow chart which shows the control procedure which changes the image data of 

#7 camera into image data from the camera in 7'1 in camera arrangement of Fig. 14. 

[Drawing 22] Drawing which explains generating of a dead angle field and double ******** in this 

operation gestalt at the time of panorama image purification. 

[Drawing 23] Drawing explaining the projection principle to the cylinder for panorama image 
generation. 

[Drawing 24] The flow chart of the control procedure concerning the modification of the control 
procedure of Fig. 10. 

[Drawing 25] The flow chart which explained a part of flow chart of Fig. 24 to the detail. 
[Drawing 26] The flow chart which explained a part of flow chart of Fig. 24 to the detail. 
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[Drawing 27] Drawing explaining the camera arrangement by other methods. 
[Drawing 28] Drawing explaining collection of image data when a car performs rectilinear- 
propagation actuation in the camera arrangement by the method of Fig. 27. 

[Drawing 29] Drawing explaining collection of image data when a car performs left-turn actuation 
in the camera arrangement by the method of Fig. 27. 

[Drawing 30] Drawing explaining collection of image data when a car performs right-turn 
actuation in the camera arrangement by the method of Fig. 27. 

[Drawing 31] Drawing explaining the camera arrangement by the method of further others. 
[Drawing 32] Drawing explaining collection of image data when a car performs rectilinear- 
propagation actuation in the camera arrangement by the method of Fig. 31 . 

[Drawing 33] Drawing explaining collection of image data when a car performs left-turn actuation 
in the camera arrangement by the method of Fig. 31 . 

[Drawing 34] Drawing explaining collection of image data when a car performs right-turn 
actuation in the camera arrangement by the method of Fig. 31. 

[Drawing 35] The flow chart which shows the creation procedure of the image database in 
connection with the 3rd modification. 
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CORRECTION OR AMENDMENT 



[Kind of official gazette] Printing of amendment by the convention of 2 of Article 1 7 of Patent 
Law 

[Section partition] The 3rd partition of the 7th section 
[Publication date] June 8, Heisei 1 3 (2001 . 6.8) 

[Publication No.] JP,1 1 -1 68754,A 
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[Procedure revision] 

[Filing Date] September 14, Heisei 11 (1999. 9.14) 

[Procedure amendment 1] 

[Document to be Amended] Specification 

[Item(s) to be Amended] 0043 

[Method of Amendment] Modification 

[Proposed Amendment] 

[0043] As the term of the conventional technique described, if these two camera stations 
estrange too much in order to generate one interpolation image from the on-the-spot photo 
image photoed in two locations of discontinuity (i.e., if camera 20g is made to estrange distantly 
[ a / camera 20] and is kept (r is set up greatly)), smoothness will be lost between a 
interpolation image and an on-the-spot photo image. Then, as this system shows to Fig. 14, it is 
the time amount width of face from #7 camera (20g). 
r/v (it is here and v is the speed of advance of a car) 

#1 thru/or #6 camera decides to use with the image data to which only r/v time amount took a 

photograph of the on-the-spot photo image data photoed when the image data of the **** 

future, i.e., #7 camera, progressed after r/v time amount in a location T on the spot in the past 

location (namely, location shown in Fig. 14). 

[Procedure amendment 2] 

[Document to be Amended] Specification 

[Item(s) to be Amended] 0049 
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[Method of Amendment] Modification 
[Proposed Amendment] 

[0049] The database generative system explained below changes into a database three logging 

files of Fig. 17 which the data collecting system of Fig. 3 generated. Generation of a database is 

performed by the image processing system computer 50 in Fig. 16. The above-mentioned video 

tape file is set to videocassette recorders 60a-60g, and is read. Moreover, PC30 is connected to 

a computer 50. 

[Procedure amendment 3] 

[Document to be Amended] Specification 

[Item(s) to be Amended] 0060 

[Method of Amendment] Modification 

[Proposed Amendment] 

[0060] It will be reflected on the central site of the image of each ****. Therefore, if the image 

of two sheets is connected. Object P is a difference between the locations in which it is 

displayed on a duplex and the image of a duplex is reflected, 

[Procedure amendment 4] 

[Document to be Amended] Specification 

[Item(s) to be Amended] 0065 

[Method of Amendment] Modification 

[Proposed Amendment] 

[0065] 

[Equation 4] 




[Procedure amendment 5] 

[Document to be Amended] Specification 

[Item(s) to be Amended] Easy explanation of a drawing 

[Method of Amendment] Modification 

[Proposed Amendment] 

[Brief Description of the Drawings] 

[Drawing 1] Drawing explaining the outline of a broader-based walk-through method in which this 
invention may be applied. 

[Drawing 2] The side elevation showing the configuration of the outline of camera loading 
equipment prepared in conventional camera arrangement equipment 
[Drawing 3] The block diagram, showing the configuration of the data collecting system 
concerning the example which applied this invention. 

[Drawing 4] Drawing showing where various data are finally recorded in the system of Fig. 3. 
[Drawing 5] The block diagram showing generation of the time code in the video camera 20 of 
Fig. 3. 

[Drawing 6] Drawing explaining a format of the data recorded on the video tape 22 of a video 
camera 20. 

[Drawing 7] Drawing explaining a format of the data recorded in the hardware disk HD of PC30. 
[Drawing 8] Drawing explaining the description of camera arrangement of this operation gestalt. 
[Drawing 9] Drawing which explains photography bearing of #1 camera thru/or #6 camera in 
camera arrangement of Fig. 8. 

[Drawing 10] The flow chart which shows the whole processing of the data collecting system of 
Fig. 3. 
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[Drawing 11] The flow chart which explains a part of flow chart of Fig. 10 to a detail. 
[Drawing 12] The flow chart which explains a part of flow chart of Fig. 10 to a detail. 
[Drawing 13] The flow chart which explains a part of flow chart of Fig. 10 to a detail. 
[Drawing 14] Drawing explaining the advantage of camera arrangement of Fig. 8. 
[Drawing 1 5] Drawing explaining the time of day of the image data unified on one record of an 
image database. 

[Drawing 16] The block diagram showing the database generation structure of a system of an 
operation gestalt. 

[Drawing 17] Fig. 6, drawing showing the configuration of files other than the format of Fig. 7. 
[Drawing 18] The flow chart explaining radical Motohara ** of control for database generation of 
an operation gestalt. 

[Drawing 19] The flow chart which explains a part of flow chart of Fig. 18 to a detail. 

[Drawing 20] The flow chart which explains a part of flow chart of Fig. 1 8 to a detail. 

[Drawing 21] The flow chart which shows the control procedure which changes the image data of 

#7 camera into image data from the camera in 7'1 in camera arrangement of Fig. 14. 

[Drawing 22] Drawing which explains generating of a dead angle field and a double field in a 

panorama image generate time in this operation gestalt. 

[Drawing 23] Drawing explaining the projection principle to the cylinder for panorama image 
generation. 

[Drawing 24] The flow chart of the control procedure concerning the modification of the control 
procedure of Fig. 10.. 

[Drawing 25] The flow chart which explained a part of flow chart of Fig. 24 to the detail. 
[Drawing 26] The flow chart which explained a part of flow chart of Fig. 24 to the detail. 
[Drawing 27] Drawing explaining the camera arrangement by other methods. 
[Drawing 28] Drawing explaining collection of image data when a car performs rectilinear- 
propagation actuation in the camera arrangement by the method of Fig. 27. 

[Drawing 29] Drawing explaining collection of image data when a car performs left-turn actuation 
in the camera arrangement by the method of Fig. 27. 

[Drawing 30] Drawing explaining collection of image data when a car performs right-turn 
actuation in the camera arrangement by the method of Fig. 27. 

[Drawing 31] Drawing explaining the camera arrangement by the method of further others. 
[Drawing 32] Drawing explaining collection of image data when a car performs rectilinear- 
propagation actuation in the camera arrangement by the method of Fig. 31 . 

[Drawing 33] Drawing explaining collection of image data when a car performs left-turn actuation 
in the camera arrangement by the method of Fig. 31 . 

[Drawing 34] Drawing explaining collection of image data when a car performs right-turn 
actuation in the camera arrangement by the method of Fig. 31. 

[Drawing 35] The flow chart which shows the creation procedure of the image database in 
connection with the 3rd modification. 



[Translation done.] 
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